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Abstract 


Schools of education are often criticized for not effectively preparing teacher candidates and 
teacher education research has been criticized for lack of rigor in examining the impact on 
student learning. Teacher preparation accrediting agencies have responded with requirements 
for programs to demonstrate the effectiveness of their graduates based on their students’ 
achievement. This has proved challenging for programs that lack access to student 
achievement data. To examine teacher effectiveness, one mid-size state university devised a 
strategy using publicly available state data. 

This case study presents an analysis of the mathematics performance of students as it relates 
to a Professional Development School program for cooperation between Laurel Woods 
Elementary School and University of Maryland, Baltimore County (UMBC). With almost 
50% of the school’s 1* through 3" grade teachers being UMBC graduates, this analysis of 34 
grade achievement data provides insight into the impact of teacher education graduates on 
their students’ achievement. The study compares Laurel Woods’ overall and subgroup data 
with that of the state. The results indicate that teacher education programs and Professional 
Development School partnerships can contribute positively to the success of schools where a 
high proportion of teachers are from one teacher education program. 


Keywords: teacher preparation, assessment of teacher performance, student learning, 
professional development schools, low-cost evaluation, impact of teacher 
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Introduction 


Recognizing that teachers play a critical role in determining student success, 
teacher education endeavors to provide prospective teachers with skills and 
experiences needed to enter the profession as highly skilled Pre-Kindergarten 
through grade 12 (PK-12) teachers and ready to maximize PK-12 student outcomes. 
However, critics often identify schools of education as the weak link in the 
preparation of teachers, arguing that teacher candidates are the lowest performance 
population in the university, take the least number of rigorous courses, and are so 
poorly prepared that fifty percent leave the profession in five years (Fraser & Lefty, 
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2018; Walsh, 2001, 2020; Tucker, 2018). Furthermore, teacher education research 
has come under fire for lack of rigor in attention to impact on student learning 
(Walsh, 2001, 2020; Tucker, 2018). Accrediting agencies such as the Council for the 
Accreditation of Educator Preparation (CAEP) responsible for the quality of teacher 
education programs responded to these criticisms with a number of policy changes 
to improve outcomes. One such change was to require teacher education programs 
to demonstrate the effectiveness of their graduates in their school settings based on 
student achievement (Darling-Hammond, 2020). The difficulty is teacher education 
programs are required to measure performance based on data which is under the 
control of a separate organization, the school district, whose concerns include 
personnel privacy, data security and student confidentiality. 

It is important that teacher education programs examine the quality of teacher 
preparation and strive for continuous improvement. Although a teacher’s success as 
demonstrated by student achievement is often considered to be indicative of her/his 
preparation program, universities have historically not been able to follow their 
graduates and review their students’ data for purposes of program improvement due 
to lack of institutionalized tracking structures and teacher and student confidentiality 
concerns (Fraser & Lefty, 2018). This leads to the question of how best to measure 
the impact of teacher education programs on PK-12 student success. 

In response to on-gong criticism of teacher education programs, the State of 
Maryland, where this study took place, committed to improving teacher education 
through a policy document entitled The Redesign of Teacher Education (1995) 
(Putnam & Walsh, 2019; Clemson-Ingram & Fessler, 1997). A core element of that 
policy proposed Professional Development Schools which were codified in 
regulations in 2007 (Clemson-Ingram & Fessler, 1997; Darling-Hammond, 2020; 
Professional Development Schools, 2007). A Professional Development School 
(PDS) is a collaboratively planned and implemented partnership between a 
university and specific PK-12 schools for the academic and clinical preparation of 
preservice teachers and the continuous professional development of both Local 
Education Agency (LEA) staff and Institutions of Higher Education (IHE) faculty. 
The focus of the PDS partnership is improved student performance through 
research-based teaching and learning. Over the years since their introduction, IHEs 
and LEAs worked to enhance the PDS experience for preservice and inservice 
teachers. Recent research on well-developed PDS partnerships suggests that 
graduates of these programs feel more confident and prepared and are rated as more 
effective by employers and supervisors, and inservice teachers who participate in the 
PDS partnership also see the collaboration as beneficial for their practice (Darling- 
Hammond, 2020). 

Standards and processes were put in place by the state to assure compliance with 
the PDS mandate through onsite reviews of teacher education programs that 
included a review of selected PDS partners using an assessment system that 
evaluated the partnership based on five standards linked to four components. The 
standards established goals for learning communities, collaborations, accountability, 
organization, and diversity. Each of these standards were examined from the 
perspective of four components: teacher preparation, continuing professional 
development, research and inquiry, and student achievement (Professional 
Development Schools, 2007). 
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Over the years since the implementation of the PDS standards, little has been 
done to assess the impact of PDS programs on student learning. This dearth of 
research reflects the difficulty in establishing a strategy for assessing the impact of 
teacher education programs on schools given the complexity and cost of the 
undertaking. Even though student achievement was the intended outcome of the 
PDS model and one of the four components of the standards, rarely did programs 
address the accountability component of the standards which is stated in two 
indicators: 

1. PDS stakeholders assume responsibility for improving PreK-12 student 

achievement. 

2. PDS partners collaborate to determine the impact of PDS on student 

achievement. 

Addressing this component of the PDS standards as well as the impact of our 
graduates is the intent of this study. 


Context 


By the year 2000, the IHEs with teacher preparation programs in Maryland had 
established memoranda of understanding with local school districts to meet the 
requirements of the PDS regulations. The University of Maryland, Baltimore County 
(UMBC) Education Department created PDS programs with multiple elementary 
and secondary schools in five school districts throughout the Baltimore-Washington 
area. Many of these PDS partners were in place by 2003 and expanded in 2007. One 
of the expansion PDS sites was Laurel Woods Elementary School (LWES), a 
suburban school near Washington, D.C., which is the subject of this study. LWES 
was selected as a PDS after discussions with the Howard County Public School 
System’s (HCPSS) Office for Professional Development Schools and LWES 
administrators and teachers who determined that the school population and UMBC 
preservice teachers could mutually benefit from the collaboration. 

LWES is a diverse school facing many challenges. In 2018, LWES served 611 
students from kindergarten through fifth grade from North Laurel, Maryland, and 
the enrollment keeps increasing, thus surpassing the maximum capacity of 609 
students which the building can hold. There are four or five classrooms at each 
grade level with an average of 22 students in each classroom. The school is a 
majority-minority school with 52.1% of the population identified as African 
American, 23.7% Hispanic, 9.5% Asian, 7.5% white, 0.3% Hawaiian, 0.3% Native 
American, and 6.6% identifying as two or more races. Approximately 60% of the 
students receive free and reduced meals, 9.0% receive services for English language 
learning, and 9.5% receive special education services. LWES is earmarked as a Title 
I school, which means it has more than 40% low-income families, and therefore 
receives additional support such as mathematics tutors, reading support teachers, and 
after-school academic support programs to improve student academic performance 
and quality of life. 

LWES started the PDS partnership with UMBC to serve the Early Childhood 
and Elementary Teacher Education programs as well as enhance education for the 
children of the Laurel Woods community. Through the collaboration, UMBC 
provides a University Liaison to work in the school one day a week providing 
professional development and support for both preservice and inservice teachers. 
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The school also selects a faculty member to serve as the PDS site liaison facilitating 
communication and professional development opportunities between the university 
and school. As a part of this partnership Laurel Woods hosts preservice teachers 
completing their 28-week early childhood and/or elementary field experiences. In 
their first semester they participate 2 days per week in one classroom with a mentor 
teacher who is jointly selected by the school and university. During the second 
semester, they teach full time in that same classroom taking over all teaching 
responsibilities for several weeks during their 80-day rotation. In addition to the 
UMBC assessment of preservice teachers, HCPSS personnel also evaluate them for 
the purpose of employing successful candidates as teachers in their schools. 

Graduates of UMBC are often employed by HCPSS and specifically at LWES 
based on their successful internship. While UMBC graduates receive positions 
across the district, during the 2017-2018 academic year, LWES employed six 
UMBC graduates of their thirteen first through third grade teachers representing 
almost half of the classroom teachers in these early grades. This high percentage of 
UMBC graduates working in one school created the opportunity to use this naturally 
occurring site for a study of program effectiveness. 

Teachers graduating from UMBC with certification in elementary or early 
childhood education receive degrees in academic fields such as psychology or 
mathematics as well as their certification. The average teacher comes from the upper 
third of all high school graduates who take college entrance examinations. Teachers 
from UMBC have course work and field experience in diverse learners, culturally 
responsive pedagogy, and methods of instruction, as well as a year-long internship 
under a mentor teacher trained in working with UMBC students. LWES continues 
the support and integration of teachers into their school so that students experience a 
consistency of methods, language, and classroom management regardless of their 
class assignment. 


Theoretical framework 


Work done by the Rand Corporation and others suggests that teachers and 
schools contribute to student learning (Opper, 2019; Xu & Swanlund, 2013; Rivkin, 
Hanushek & Kain, 2005). Further, coherent, long-term professional development is 
demonstrated to be an effective model with extensive practice after implementation 
leading to a positive, long-term impact on teacher performance and _ student 
achievement (Yoon et al., 2007; Polly et al., 2018). To the UMBC faculty, this 
strategy of long-term practice for professional development is modeled by 
Professional Development Schools such as LWES where a long-term, coherent and 
developmental program is the model for inducting teacher candidates into a school 
setting and enhancing skills of experienced teachers in the school. Furthermore, the 
nature of mathematics curriculum usually means that it is learned at school, whereas 
other areas, like literacy, may demonstrate more of a parental or family influence 
(Sonnenschein, Stites & Dowling, 2020). Therefore, this study examines 
mathematics scores because it is more indicative of teacher influence. 

As noted in the Rand report, “many factors contribute to a student’s academic 
performance, including individual characteristics and family and neighborhood 
experiences. But research suggests that, among school-related factors, teachers 
matter most. When it comes to student performance on reading and math tests, a 
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teacher is estimated to have two to three times the impact of any other school factor, 
including services, facilities, and even leadership” (Opper, 2019, p. 1). However, it 
is important to consider that the teacher performance is nested within the school and 
that it is difficult to separate the contribution of the teacher to school performance 
and the school’s influence on the teacher (Xu & Swanlund, 2013). This is supported 
by recent research from Ronfeldt et al. (2018) that indicated one major element of 
success of preservice teachers is employment in the school where they had interned. 


Methodology 


An impact study was designed in three phases to examine the effect of the PDS 
program and UMBC graduates on student learning. The three-phase design reflects 
the complexity of examining teacher performance, the availability of data sources, 
the questions addressed in the study, and the potential for cost or privacy concerns. 
Not all phases have been completed at this time. Each phase of this study contributes 
a perspective on the school and student learning that, when brought together, 
provide a more complete understanding of teacher effectiveness and the impact of 
the teacher preparation program and PDS partnership. Phase I, the focus of this 
study, utilizes public records available to the university researchers on state and 
local websites and related institutional reports from HCPSS. Phase II includes 
interviews and focus groups with teachers, administrators, and UMBC personnel 
assigned to the school. Phase II involves data requested in Fall of 2015 by the 
UMBC Department of Education, but data on individual teachers’ evaluations or 
their specific students’ performance were not made available by the school district 
for privacy reasons. This is an ongoing conversation with the school district, but in 
the meantime, this made the public data even more essential to understanding 
UMBC's contribution to student learning. 

The Phase I strategy was undertaken in spring of 2018 using publicly collected 
data that was part of state-mandated assessments. This made the data for the Phase I 
study both a low-cost and an unobtrusive strategy. Data collection was conducted by 
the school as part of its yearly assessment, making it a no-cost method to determine 
effectiveness. The study did not require HCPSS or UMBC Institutional Research 
Board approval as data was available from the district and state websites. The study 
did not intrude on the school routines or reduce instructional time and, finally, it 
responded to criticism leveled at the lack of performance data of graduates of 
teacher education programs while meeting CAEP national accreditation standards 
for linking standardized testing outcomes of students to our graduates’ instruction. 

The UMBC research team proposed studying the performance of students in 
third grade rather than all grades because UMBC’s graduates contributed 
extensively to student learning and the student test performance in the first three 
grades at the school and public data was available beginning in grade three. The 
research team believed findings on the school performance then could, in part, be 
attributed to the effectiveness of UMBC’s graduates and the teacher education 
program. The research team recognized that specific teachers could not be linked to 
specific student outcomes as the public data is reported at the grade level rather than 
the classroom level. However, drawing on teacher effectiveness research and school 
effectiveness research, the research team concluded that the teachers acting together 
can be both individually effective and, more importantly from a school and student 
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perspective, can be an effective instructional team. The research team also 
recognized that the school environment contributes to the teachers’ 
accomplishments just as teachers enhance the school’s success. 

Teacher performance data as measured by value added methods is considered by 
CAEP as the standard for assessment of Education Provider Programs. Some states 
have that data available through a statewide assessment system. Maryland does not 
have that kind of a system in place nor do they currently have systems that permit 
the tracking of teacher candidates once they have left their certification program. 
Without ready access to teacher-by-teacher student performance data, the UMBC 
Education Department needed to come up with alternative means of assessing 
graduate and program impact on PK-12 student learning using mixed methods and 
multiple data points. Public domain data was collected in June of 2018 to establish 
baseline information on the school and to determine what questions might be most 
appropriate to ask university personnel, teachers, and administrators in later studies. 
With test performance on a teacher-by-teacher basis not available currently in 
Maryland, UMBC uses school-level data as a proxy by examining test performance 
by grade level and subpopulations within the school. These findings are then 
compared to state level data. 

The teachers included in this study were UMBC graduates of the early 
childhood or elementary programs and full-time teachers at LWES in grades one 
through three. Three out of five teachers at the first-grade level, two out of four 
teachers at the second-grade level, and one out of four teachers at the third-grade 
level were UMBC alumni. The data presented in this study examine third grade 
because that is the grade level in which test data is available. The third-grade scores 
reflect learning at the first and second grade as well as third grade, and can, in part, 
be attributed to those teachers. There are no state or nationally normed tests 
available to the researchers for students in first and second grade. Upper grade 
scores were not reviewed as Laurel Woods has not employed a large number of 
UMBC teachers in those grade levels. 

The data analyzed came from assessments designed for grades 3-8 by the 
Partnership for Assessment of Readiness for College and Careers (PARCC). 
PARCC was specifically designed as a means of measuring student achievement of 
the Common Core State Standards (CCSS) in English Language Arts and math. The 
tests provide students with a scaled score that is then used to place them in one of 5 
achievement levels to determine if they have met or exceeded standards: Level 1 
“Did Not Meet” expectations, Level 2 “Partially Met” expectations, Level 3 
“Approached” expectations, Level 4 “Met” expectations, and Level 5 “Exceeded” 
expectations (more info on PARCC can be found at _ http://mdk12- 
archive.msde.maryland.gov/assessments/parcc/index.html). For the purposes of this 
study, we will be comparing the percentage of students in the school and the state 
that met and exceeded expectations in math. 

This study compares LWES overall student population scores with state overall 
student population scores as well as examining subsets of the student population to 
determine if the school is meeting a major goal of the school district and the state to 
reduce achievement gaps between subgroups. For LWES, the subgroups that we are 
focusing on are African American, Hispanic, and Free and Reduced Meal 
subpopulations because these groups are ones that are of concern. 
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Results 


An analysis of LWES scores in comparison with the other schools in the state 
demonstrates that it is a successful school. The LWES African American population 
percentile at the Exceeded level (6.1%) almost doubled the African American State 
percentile at the Exceeded level (3.7%), while LWES African American students at 
the Met level (32.7%) were almost 10 percentage points higher than the state 
African American students (22.4%). The overall percentile for African American 
students who Met or Exceeded expectations at the state was 26% while at LWES it 
was 38.8%, which is over 12 percentage points higher than the state. Similarly, the 
percentage of African American students at the state who achieved at the Did Not 
Meet level was 22.6%, while at LWES it was only 10.2%, which is 12 percentage 
points lower than the state African American population and 4.1 percentage points 
lower than the overall state population. 

The percentage of LWES Hispanic population that Exceeded or Met standards 
according to PARCC also far surpassed the state Hispanic population with 12.5% 
versus 4.6% in the Exceeded category and 29.2% versus 23.7% in the Met category. 
Overall, 41.7% of Hispanic students at LWES Met or Exceeded standards compared 
to 28.3% of Hispanic students across the state. The percentage of students in the Not 
Met category at LWES (8.3%) was less than half of that for the Hispanic students 
across the state (19.6%). In fact, the percentage of the Hispanic student population at 
LWES who Met or Exceeded standards (41.7%) was similar to the percentage of all 
student populations across the state (43%). 

Furthermore, LWES has a Free and Reduced Meals (FARMS) population of 
approximately 60%. FARMS is often used as a proxy for determining low 
socioeconomic status (SES) in a school. According to previous studies, low SES can 
result in an 18% difference in scores compared to a high SES population (Baird, 
2012). Baird argues that the difference between high and low SES performance is 
one standard deviation. Based on the low SES of the majority of students at LWES, 
it would be expected for them to have scores significantly lower than the state 
average. 

However, our findings indicate that LWES had only 0.4% fewer students on 
average at the Exceeded level and is actually 3.2% higher at the Met level. Overall, 
the LWES percentage of students that Exceeded standards was 45.7% compared to 
the overall state population at 43.0%; the LWES population was 2.7 percentage 
points higher than the state, and well above Baird’s estimation for performance. Of 
equal importance, LWES has only 9.5% that Did Not Meet standards while 14.3% 
of the state population are in the category of Not Meeting the standards, closing the 
gap at both ends of the performance measures. 

Given research that identifies teachers as a salient contributing factor in student 
outcomes, this examination of student achievement suggests the effectiveness of 
teachers prepared through the PDS model as well as their positive impact on student 
achievement. Students from sub-populations who often underperform on 
standardized assessments did better at LWES than their counterparts across the 
states. 
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Discussion 


This study compares performance of third graders at LWES to those across the 
state of Maryland. While these are not statistically equal groups on any measure, all 
members of the population were included. In the end, third grade students at LWES 
outperformed their peers from other schools, including those with lower percentages 
of FARMS and ELLs. LWES students’ strong performance in math across all 
subgroups and economic conditions must credit the school’s effective instruction. 
The authors recognize that no one single lever will improve a school’s academic 
performance. However, the research literature supports that teachers are a major 
contributing factor in student achievement and that long, coherent training and 
practice is an effective strategy to improve teacher performance and student 
learning. The contributions of the school to individual teacher’s performance and 
individual teacher contributions to overall school performance is difficult to 
separate, but the findings are encouraging that available measures can establish 
linkages among cadres of teachers and their students’ performance. 

The findings support both the value of PDS programs and UMBC education 
programs in the superior performance of the school against similar populations state- 
wide. The contribution of PDS programs through the involvement of interns prior to 
their employment at the school and the long-term development of a teaching staff 
through the internship process yields substantial success in comparison to similar 
student populations throughout the state of Maryland. The PDS model offers a 
pathway for teacher development and school achievement, and the PDS standards 
require assessment of PDS impact on student achievement that is often difficult to 
assess. The use of publicly available data from the school forms a basic if imperfect 
source for developing reviews of programs. The development of a longitudinal 
database now being created by the state may offer additional data for comparisons 
both within and across schools with similar programs or similar demographics. Until 
then, this type of analysis provides some insight into the potential benefit of the PDS 
model for student outcomes. 


Policy implications 


While we agree with our accrediting bodies that it is ideal for educator 
preparation programs to assess the effectiveness of their graduates, we know first- 
hand the challenges of doing so within our current system. Professional 
Development Schools not only provide preservice teachers with rich, supported 
teaching experience and IHE faculty with opportunities to better understand the 
current context of teaching, they also provide a potential avenue for studying our 
graduates’ impact on student learning. The UMBC Department of Education will 
discuss these findings with our PDS partners and encourage them to develop 
extended, qualitative and quantitative analysis in collaboration with the department. 


Future studies 


While it would be preferable to compare the school’s math scores from before 
the establishment of the partnership to after its implementation, it was unfortunately 
impossible to do so because the PARCC assessment was not in use when the 
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partnership was established in 2007. There are clear limitations to using student 
achievement data as the sole means of evaluating teachers and education preparation 
programs (Darling-Hammond, 2020). In addition to being difficult to access the 
needed data, many have called into question the ability of standardized tests to 
adequately capture all that students learn and teachers teach. This is why an in-depth 
qualitative study of the internship and beginning years of teachers at LWES as 
outlined in the remaining phases of our study is necessary to provide more specific 
findings that could support and expand our understanding of the potential impact of 
PDS on teacher preparation. Additional studies of other settings where UMBC has a 
major presence could determine if these findings are consistent across other schools. 
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